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Executive Summary 


Citizens Advice has seen a 63 per cent increase in requests for help to complain 
about public services over the last four years. We decided to explore why people 
are struggling to complain, or deterred from complaining, and how reversing these 
trends and making better use of complaints data could drive service improvements. 
We found that: 


People don’t make formal complaints after poor public service. 45 per cent of 
people have experienced poor public service recently, but only 22 per cent of them 
went on to make a formal complaint. However, 73 per cent used informal channels 
to share their frustrations, with one in five younger people turning to social media, 
for instance. 


The feedback and insight of specific groups is being missed. 52 per cent of 
people said they didn’t complain about a poor experience with a public service 
because they felt ‘it wouldn't change anything’. Younger and older people, and 
those on lower incomes, tend to find it hardest to make formal complaints. 


Aggregating complaints data, including from informal channels such as social 
media, can provide important insight into potential service improvements. 
These insights - for example on call waiting times to HMRC - can be gained and 
acted on in real time. 


Mistakes happen, but we are wasting opportunities to improve public services if 
people are put off complaining and if the value of complaints data is not realised. 
Through our research, we identify six key opportunities: 


@ There should be no ‘wrong door’ when people want to make a complaint. 

@ Public bodies should develop better insight and intelligence capabilities on 
emerging service issues through big data aggregation and other means, to 
make services more effective and responsive to public needs. 

Public service providers need to modernise their complaints culture and 
empower frontline staff. 

People should have access to an independent, impartial organisation for help 
with their complaint. 

People should be able to complain to a single point of contact. 

The power to raise ‘Super complaints’ on behalf of public service users should 
be introduced. 


Methodology 


This report combines insights from several sets of quantitative and qualitative data. 
We draw upon intelligence from our own network, looking at data recorded about 
clients of local Citizens Advice across the country, including trends in ‘advice issue 
codes’ (AlCs) for England, which are recorded by our advisers each time they help a 
client. We also draw upon qualitative data collected in a series of five 
semi-structured depth interviews and three focus groups with local Citizens Advice 
clients who have gone through public service complaints processes.' We combine 
this with insights from the Citizens Advice Network Panel. The Network Panel is a 
monthly survey sent to over 800 staff and volunteers across England and Wales, 
asking about their experiences of and views on policy issues.’ 


Apart from Citizens Advice network and client data, we analyse throughout the 
report the results of two nationally representative social surveys we commissioned. 
The Complaints Survey? results are based on a weighted sample of 1,719 adults 
resident in England, surveyed in December 2015, and the Civic Life Survey‘ results 
are based on a sample of 2,025 UK adults, surveyed in December 2014. 


Finally, this report analyses publicly available data collected from Twitter. Given the 
sheer scale of the ‘big data’ on offer from social media sites, systematic analysis is 
beyond the capacity of individual human researchers. We turned instead to natural 
language processing (NLP) software, which allows researchers to engage in a form 
of interactive and iterative ‘machine learning’ so that algorithms can be used to 
reliably and automatically categorise very large numbers of texts. Working with 
software developed by the Centre for Analysis of Social Media (CASM) at Demos 
and the University of Sussex, Citizens Advice researchers were able to analyse the 
Twitter data for our case study services. We looked at all ‘at mentions’ (where a user 
tags another user's handle in a tweet by including, for example, ‘@CitizensAdvice...’) 
of @HMRCgovuk, @DVLAgovuk and 345 separate English local authority Twitter 
handles, over the course of one year; a total of 441,172 tweets. Using this 
methodology, we were able to develop algorithms capable of identifying 
complaints, as well as types of complaint, from large datasets with a minimum 
accuracy level of 70 per cent. 


1 We interviewed local Citizens Advice clients about their experiences of making complaints about poor 
public services over the period November 2015 - January 2016. This included complaints about a 
hospital, HMRC, a school, the DWP and a social care complaint to a local authority. The focus groups 
took place on 1, 2 and 14 December 2015. 

* 278 staff and volunteers from across the national network responded to this round of the Network 
Panel survey. 

3 The Complaints Survey was carried out by Populus. 

4 The Civic Life Survey was carried out by Boxclever Consulting. 


Chapter 1. Context 


Growing recognition of the need for reform 

Reforms to the public service complaints landscape are long overdue. Government, 
key service providers, and the various public service ‘ombudsmen’ charged with 
handling top level complaints, have all suggested that the existing complaints 
landscape is in urgent need of an overhaul. 


The clinical failings that took place in Stafford Hospital highlighted the vital need to 
get better at capturing and learning from complaints. The 2013 Francis Inquiry 
identified as a key driver of those problems the failure of the local NHS Trust to 
listen to, and act upon, complaints from service users and their families.” This 
triggered a series of further investigations and reports from public service 
Stakeholders including a government mandated review into NHS complaints 
handling. In the Parliamentary and Health Service Ombudsman (PHSO) submission 
to that review, Dame Julie Mellor spoke of a “toxic cocktail” of reluctance to 
complain on the part of service users and unwillingness to listen to complaints on 
the part of NHS services.° 


While it was an NHS service that first drew serious attention to the question of 
public service complaints handling, the debate has quickly expanded to encompass 
public services more broadly. The House of Commons Public Administration Select 
Committee (PASC) report More complaints please! was also prompted by the 
Stafford Hospital case, but MPs used the report to set out a rights-based approach 
to public service complaints, wherein people have the “right to have their complaint 
respected, for their concerns to be addressed, and for any wrongdoing to be put 
right with fair redress and without undue delay”.’ 


But despite growing levels of scrutiny of current public service complaints systems, 
reforms have been limited. As a report by Healthwatch England explained, “change 
so far has only tinkered with the existing bureaucratic arrangements for complaints 
handling, and looked largely at things from the system's perspective”.® Frustrations 
with the system, however, are gaining momentum. Following the Gordon Review, 
the Government has committed to publishing a Draft Bill which will bring together 
the public service ombudsmen. Whilst the content of the Draft Bill is not yet 
confirmed, its aim will be to simplify and improve the current complaints process.? 


> Thomas Powell (2013), The Francis Report (Report of the Mid-Staffordshire NHS Foundation Trust 
public inquiry) and the Government's response. 

e PHSO (2013), Submission by the Health Service Ombudsman for England to the Review of the NHS 
Complaints System. 

” PASC (2014), More complaints please! Twelfth Report of Session 2013-14. 

8 Healthwatch England (2014), Suffering in Silence. 

? HM Government (2015), A Public Service Ombudsman: Government Response to Consultation. 


At Citizens Advice, we see this widespread recognition of the importance of 
capturing complaints as a valuable opportunity to accelerate improvements, as well 
as prevent service failings. 


Citizens Advice data on public services 

In the last year Citizens Advice provided face-to-face and phone advice to 2.5 million 
people from 2,500 locations in England and Wales, helping with 6.2 million advice 
issues.'° We record these issues to three tiers of detail, from a broad category like 
‘health’ to a specific problem like ‘registration with GPs’. We can analyse this data by 
postcode and often demographics and income. 


We have seen a slow but steady increase in the number of queries about public 
services over the last four years, from an annual total of 104,674"' to 116,790;'* an 
increase of 12 per cent. Public services cover a number of advice categories across 
health (hospitals, general medical practice and dentists), education (schools), 
housing (local authority housing and homelessness services), social services (local 
authority) and tax issues. 


Within this increase in public service queries, as Figure 1 shows, we have also seen a 
63 per cent increase in the number of people coming to Citizens Advice for help 
making complaints against a range of public services, with 13,038 complaints 
queries recorded in the last year. We see ‘customers’ of public services struggling to 
get the services they need, and then struggling to complain when they don't. 


Figure 1: Trend in public service complaints queries, Q4 2011-12 to Q3 2015-16% 
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10 See Citizens Advice (2015), The value of the Citizens Advice service: our impact in 2014/15. 
11 Q4 2011-12 to Q3 2012-13, England only. 


12 Q4 2014-15 to Q3 2015-16, England only. 
13 Based on Citizens Advice advice issue code data for England only. 


Particular public services for which complaints queries have increased significantly 
in this period include schools, hospitals, GP practices, dentists and local authority 
homelessness services. '* 
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We gain further insight into public service complaints as partners of Healthwatch, 
the consumer champion for health and social care. Local Healthwatch consumer 
services were delivered through more than 80 local Citizens Advice offices in the 
last year, and Healthwatch England uses the data gathered to identify and 
comment on emerging issues.'? In our capacity as providers of Healthwatch 
services, we helped clients with 14,028 advice issues in the last year. 


14 These individual complaints issue increases - represented in the infographic - include England and 
Wales. 


1> Healthwatch (2014), Suffering in Silence. 


Chapter 2. Poor experiences with 
public services don’t translate 
into formal complaints 


“The majority of people are so busy living their lives. Initially they might be 
really angry or upset and be full of passion for pursuing this. And after a few 
weeks of just being messed around, they just give up. That's exactly what 
they want us to do and that's exactly what the plan is. They know if they 
mess people around for long enough that they'll just give up. And 
unfortunately most people do because they've got lives to lead and things 
that they'd rather be doing than just complaining.”'® 


In this chapter, we examine how people register their frustration when they have a 
poor experience with a public service. We wanted to understand how people 
respond when they feel let down by their local GP surgery and how they express 
their dissatisfaction when they can't get through to HMRC. We know that many 
public services see low numbers of formal complaints. NAO research into public 
service complaints notes, for instance, that some social care providers receive “few 
complaints (often fewer than 5 a yearn”. 


We see quite a different picture in the private sector. We know, for example, that in 
the last year the Financial Ombudsman Service, which handles top level consumer 
complaints against banks, mortgage lenders, insurance companies, ‘payday’ loan 
and other private finance providers, received 45 enquiries for every one received by 
the two main public service ombudsmen combined. '® 


Does this mean people are receiving excellent service from public services? Or does 
it mean that people are simply avoiding making formal complaints? In which case, 
are they turning to other channels to express their dissatisfaction? 


To better understand this issue, we commissioned a nationally-representative 
Complaints Survey, asking people about their recent experiences with key public 


16€ Complainant focus group participant and Citizens Advice client. 

17 NAO (2015), Public service markets: Putting things right when they go wrong. 

18 The Financial Ombudsman Service received 1,786,973 enquiries in 2014/15, the Parliamentary and 
Health Services Ombudsman (PHSO), which deals with complaints about GPs, hospitals, and other NHS 
services as well as government departments like the DWP received 29,000 enquiries, and the Local 
Government Ombudsman (LGO), which is responsible for top level complaints about local councils and 
the services they offer, from social care providers (including independent and privately operated ones) 
and housing to refuse collection and libraries, received 11,094. 
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services, and about complaints. Our survey found that many people had been 
through a poor service experience with a public service recently. Nearly one in two 
(45 per cent) reported that they had “experienced poor service in the last two years 
when dealing with a public service provider”; this could include as many as 19 
million people.'? 


Figure 2: Services people have had a poor experience with in the last two 
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As Figure 2 shows, of those who reported a recent poor service experience, more 
than a third (37 per cent) had a poor experience with a GP service, while more than 
a quarter had experienced poor service from a local authority (29 per cent) or 
hospital (28 per cent) and one in five (20 per cent) had experienced poor service 
from HMRC. 


Poor experiences don't translate into formal complaints 

Despite the high frequency of unsatisfactory public service experiences, we found 
that relatively few people actually go on to make a formal complaint. As Figure 3 
shows, of those who had experienced poor service within the last two years, fewer 
than one in four (22 per cent*') told us that they went on to make a formal 


19? Based on ONS population estimate of 42,724,917 adults in England. 

2° Responses based upon the 45 per cent of Complaints Survey respondents who said they had 
“experienced poor service in the last two years when dealing with a public service provider”. 

*" This is the net figure for all those who answered either “I made a formal complaint to the public 
service provider” or “I made a formal complaint to the public service provider and then escalated my 
complaint to an independent”. 


complaint about the experience.** This means as many as 15 million people may 
not have followed up their poor experience with a formal complaint.” 


Figure 3: Proportion of people who experienced poor public service in the last 
two years and made formal complaints” 
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This finding is supported by the Citizens Advice Network Panel. More than one in 
two Network Panel respondents believed their clients would find it either ‘quite 
difficult’ or ‘very difficult’ to make a complaint about an NHS service (56 per cent) or 
local authority (53 per cent) and more than three-quarters said the same for 
JobCentre Plus (79 per cent) and HMRC (84 per cent). 


*¢ Our research findings here correlate fairly closely with some of the insights offered by the PHSO’s 
2015 survey on public service complaints, which found that 34 per cent of people who experienced 
poor public service in the past 12 months went on to complain. However, whereas the PHSO found 27 
per cent of respondents to their survey had experienced poor public service in total recently, our 
Complaints Survey - which asked respondents to tell us about experiences over a longer period, 24 
months - found that 45 per cent had experienced poor service. 

?3 Based on ONS population estimate, 45 per cent of the adult population of England (proportion 
experiencing poor service) = 19,226,213 of which 22 per cent (4,229,767) did complain formally and 
14,996,446 (78 per cent) did not. 

“ Responses based upon the 45 per cent of Complaints Survey respondents who said they had 
“experienced poor service in the last two years when dealing with a public service provider”. 


Figure 4: Network panel results: Ease or difficulty for Citizens Advice clients to 
make a complaint” 
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People do complain, but not always through formal channels 

People are often reluctant to complain through formal channels after a poor public 
service experience, but this is not to say that people don’t complain at all. In fact, 
we found that many people complain through less formal channels, with only a 
relatively small proportion (17 per cent) taking no action at all as a result of their 
experience. 


As Figure 5 shows, of the people who told us that they had experienced poor public 
service from a provider in the last two years, nearly one in two (46 per cent) 
discussed their experience with friends or family, while around one third (29 per 
cent) spoke about their experience over the phone with staff at the service in 
question, and one in five (20 per cent) discussed their experience face-to-face with 
a member of staff at the service. Overall, we found that 73 per cent used informal 
channels to voice their frustrations, while 60 per cent exclusively used informal 
channels.*° We also see digital channels emerging as popular methods of 
expressing frustration after poor public service, with social media and website or 
email contact with the public service each being cited by one in ten (11 per cent) 
respondents as the channel they used to discuss a poor experience. 


*° Based on all responses to the Citizens Advice Network Panel survey. 
26 60 per cent is the net figure based upon all unique respondents who selected any response(s) 
excluding both formal complaint responses and ‘I took no action’. 


Figure 5: Action taken following a poor public service experience?’ 
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But despite the popularity of informal complaints channels, most public services 
remain ill-equipped to capture intelligence from these sources. As one Citizens 
Advice client explained: 


“All this paperwork is so outdated now, in the modern day that we live in. 
| honestly think if you've got an e-mail address you should be able to do 
things through e-mail because you can do things through their website if 
you like, you know you can do it online with the new Tax Credits form and 
whatnot, so you should be able to get in touch with them through the 
Same website if you like. But you just can’t, and they don't do emails 
either. The thing is you could speed up the process so much by doing 
that, you know?” 


In the next chapter, we explore in greater detail how existing formal complaints 
processes can make it especially difficult for some people to make their public 
service complaints heard. 


*” Responses based upon the 45 per cent of Complaints Survey respondents who said they had 
“experienced poor service in the last two years when dealing with a public service provider”. 
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Chapter 3. The feedback and 
insight of specific groups is being 
missed 


“We felt that nobody else could have done a great deal more than we did. 
They may have had a bigger stick than an independent person, but we 
weren't a pushover, we weren't letting things ride. God knows how it would 
happen if you weren't any good at writing letters or emails. My 
mother-in-law couldn't have done any of this for herself, it’s impossible, 
she’s 90 years old.”8 


Mistakes will always happen. People live messy, complicated lives and things in the 
real world do not always go according to plan. Public services are no exception. But 
when things do go wrong with public services people should have simple and 
accessible ways to register their frustration and contribute to improving the service 
for the future. 


The previous chapter showed that, while poor experience of public services is 
common, few people go on to register this experience through a formal complaint. 
In this chapter, we explore who it is that does complain, and the reasons why 
others are not doing so. 


Who formally complains? 

We looked at the age profile of the 22 per cent of people who said they are willing 
and able to formally register their frustration after experiencing poor public service. 
People in this group are more likely to be aged 35-64, as Figure 6 shows. 


28 Citizens Advice client and interviewee. 


Figure 6: People who made a formal complaint after experiencing poor public 
service, by age” 
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Previous Citizens Advice research” also found that those with the highest 
household income are most likely to say that they would find a way to share their 
view if they experienced poor public service: almost half (44 per cent) of those with 
a household income of over £90,000 answered in this way,?' compared to just less 
than a third (32 per cent) across the whole income distribution. The PHSO has 
recognised that those on lower incomes are less likely to be aware of how to 
complain about poor service, which may help to explain why this group is more 
likely to come to Citizens Advice for advocacy support. One in four (26 per cent) of 
those who come to us for advice about a public service complaints issue are from 
the lowest income bracket, compared to only 7 per cent from the highest.” One 
possible implication of this is that we could be missing out on insights on particular 
services. The PHSO has found that health services account for four out of five (79 
per cent) of complaints it receives, suggesting that other government services 
within its remit may be under-represented.** 


Why do most people avoid formally complaining? 
Four out of five people (78 per cent) told us they avoided making a formal 
complaint when they had a poor experience with a public service. We found that 


*° Responses based upon the 22 per cent of those Complaints Survey respondents who said they 
“made a formal complaint” after their last poor public service experience. Total adds up to more than 
100 per cent as respondents could select more than one answer. Making a formal complaint includes 
complaining formally to a public service provider and/or an independent body. 

30 Citizens Advice (2015), Going with the grain: Why democracy needs to fit with modern life. 

31 It should be noted that the numbers of respondents to our Civic Life Survey with a household 
income over £90,000 were small: only 55 people were in this group. 

32 PHSO, Complain for change. 

3 Based upon 2,853 unique complaints client records - where the client supplied income information - 
for the last year (Q4 2014-15 to Q3 2015-16). Lowest household income bracket is coded as <£400 pcm, 
highest as >£1999 pcm. While Citizens Advice clients are in general almost five times more likely to live 
on alow income than an average member of the UK population, in the same period, the proportion of 
clients in the lowest bracket across all advice issue areas was 22 per cent, with 9 per cent in the 
highest. 

%4 PHSO, Annual Report and Accounts 2013-14. 


only one in five (19 per cent) said this was because they were able to resolve their 
problem without needing to complain. As Figure 7 shows, a variety of less positive 
reasons explain why people do not make a formal complaint when they have a 
poor experience with public services. 


Figure 7: Reasons for not complaining after experiencing poor public service 
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Using evidence from our Complaints Survey, our Civic Life Survey, and interviews 
with Citizens Advice clients and advisers, we were able to explore people’s attitudes 
towards the complaints systems in more depth. We identify four broad reasons 
which offer insight into why people don't complain. 


1. Many people don't think their complaint will be heard and taken seriously 
Over half (52 per cent) of respondents to our Complaints Survey said they did not 
complain because they ‘didn’t think it would change anything’. As one Citizens 
Advice client commented: 


“Speaking to the council is like speaking to a brick wall. They just need to 
listen. They need to use the advice the public can give them to improve 
things and do things right instead of wasting money and doing things 
wrong.”°> 


We also asked people what would help them to make a formal complaint against a 
public service provider in the future. 69 per cent said it was ‘knowing that my 
complaint would be taken seriously’, which suggests that people are not safe in this 
knowledge at the moment. Older respondents are more likely to suggest that 
increased faith in the system would help them complain; 75 per cent of over 65s 
felt this way, compared with 61 per cent of 18-24 year olds. Older people are also 


35 Citizens Advice client and interviewee. 


more likely to say that advocacy support would help them to feel that they would 
be heard and taken seriously. While one in four (26 per cent) respondents to our 
Complaints Survey say ‘Support from an advice or advocacy organisation’ would 
help them to complain in the future, this rises to almost a third (32 per cent) 
amongst the over 65s. 


2. People don't always feel confident enough to make a formal complaint - 
especially those in older age or on lower incomes 


Previous Citizens Advice research has explored the relationship between different 
personality types and attitudes to engagement with civic and public life.°° As Figure 
8 shows, we found that people who consider themselves less assertive are also less 
likely to think that systems for complaining about public services are working well. 


Figure 8: People who think the current systems for making a complaint about 
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Responses to our Complaints Survey confirmed the importance of assertiveness 
when it comes to formally registering your frustration about poor public service. 


Amongst those who had experienced poor public service in the past two years but 


did not complain, 19 per cent highlighted worries about how the public service 
provider would treat them afterwards as their reason for not feeding back about 
their experience. This rises to a quarter (26 per cent) amongst those with a 
household income of up to £14,000, compared to just 7 per cent amongst those 
with a household income of £41,001 or more. 


For those on lower incomes, who may be more reliant on public services, 
complaining about poor public or council service can seem a forbidding prospect. 
As one Citizens Advice client put it: 


36 Citizens Advice (2015), Going with the grain: Why democracy needs to fit with modern life. 


3” Responses based on the 34 per cent of respondents to the Civic Life Survey who said that the 
systems for making a complaint about a public or council service are not working well and need to 
improve 


“There is a weird connection between certain things. You think if you 
complain about something it might impact on something else that you are 
claiming. If you're complaining say about the Jobcentre will it have an impact 
on how they decide on your case in the future? Will it have an impact on how 
you are treated? It's not very clear whether things are confidential.” 


Four out of ten (41 per cent) respondents to our Complaints Survey highlighted 
‘Knowing | would not be treated differently by the public service provider as a result 
of my complaint’ as a way to help them to complain about poor public service in the 
future. Older people were more likely to see this as important; 47 per cent of over 
65s said this would help them complain in the future, compared to 37 per cent of 
18-24 year olds. 


3. Not everyone knows how to make a formal complaint 

Providing insight which could help improve public services for the future should be 
Simple and straightforward. But this is not always the case with the current 
complaints landscape. 


Among our non-complainants, 14 per cent said they ‘did not know how to make a 
formal complaint’. As Figure 9 shows, younger people are more likely to cite 
insufficient knowledge as the reason for not complaining: 23 per cent of 18-24 year 
olds gave this reason, falling to 6 per cent amongst the over 65s. 


Figure 9: Percentage who didn’t complain because they didn’t know how, by 
age? 
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These findings correlate with previous research which has highlighted low 
awareness of ombudsmen services.” Furthermore, even when people are aware of 


38 Responses based upon the 14 per cent of those Complaints Survey respondents who said they didn’t 
make a formal complaint after their last poor public service experience because ‘I did not know how to 
make a formal complaint’. 


3? NAO (2015), Public service markets: Putting things right when they go wrong. 


the different ombudsmen, they are frequently uncertain of the process. Of the 
29,000 new cases the PHSO handled last year, the majority (66 per cent) were 
referred back to other organisations or for local resolution (which must be 
exhausted before an ombudsman can consider a complaint), suggesting a 
widespread lack of knowledge of how the system works.”° A top level complaints 
system which ‘bounces back’ the majority of people is not only frustrating, but also 
costly and inefficient. 


4. Making a formal complaint can be too complicated and time consuming, 
especially for younger people 

Knowing how to make a formal complaint does not necessarily mean you'll be 
encouraged to make one. Many think formally complaining will be too 
burdensome. Amongst the respondents to our Complaints Survey who had 
experienced poor public service in the past two years but did not complain 
formally, 18 per cent said they ‘thought it would be too complicated’. This rises to 
29 per cent amongst 18-24 year olds, the highest of all age groups. 


The length of time it takes to complain also puts people off from complaining. 15 
per cent of respondents said they avoided making a formal complaint about a poor 
experience with a public service because they thought it would take too long. Again, 
18-24 year olds were the age group most likely to say that they did not complain 
because it would take too long, with 30 per cent giving this reason. 


As a Citizens Advice adviser responding to our Network Panel Survey remarked: 
“Complaint procedures are complicated for clients to understand and take far too 
long.” One Citizens Advice client we interviewed captured the sense of irritation 
generated by the formal complaints procedure particularly well: 


“AS soon as you Start engaging in... complaining to a council, all of a sudden it 
becomes a full-time job because they know all of the loopholes and they can 
twist you back and forward and have you run all over the place looking for 
your own information.” 


Furthermore, escalating a complaint can also be difficult. For example, after 
exhausting local resolution, a complainant must contact their MP in order to 
register a (non-health) complaint with the PHSO. This can be very frustrating. As 
one Citizens Advice client told us: 


“| rang the ombudsman and | said ‘look, what happens if they've been so slow 
at resolving this, or at least even looking at it, surely you must be able to...’ 


1 


and he said ‘no’. 


4 PHSO (2015), The Ombudsman's Annual Report and Accounts 2014-15. 


The PHSO itself recognises that “direct access is generally seen as an essential 
feature of a modern ombudsman institution”, while “not being able to access an 
ombudsman directly is seen as having an adverse impact on a complainant”.*' 


The diversity of reasons people give for not formally registering their frustration 
means that, in order to better harness the insights of all people, public service 
providers need to look to multiple channels for people to express dissatisfaction 
after being poorly served and explore new, untapped sources of complaints. We 
turn to this in Chapter 4. 


41 PHSO (2011), Consultation on direct access to the Parliamentary Ombudsman. 


Chapter 4. Untapped potential 
and new sources of complaints 
intelligence: case studies from 
social media 


“And really what social media does, | think, is it turns it on its head because 
in the past it's always been that councils and big corporations have the 
power to bamboozle you. Whereas you can turn the volume of people into 
the problem for them. | think it turns the tables a little bit”’.* 


This chapter explores the potential benefits of aggregating data from a range of 
complaints channels. These include identifying trends in consumer experience of a 
particular public service and using this analysis to drive improvements, as well as 
prevent service failures. 


There is currently no aggregation of complaints data across the public service 
ombudsmen, and this aggregation is not something the ombudsmen are equipped 
to do. The NAO found “no standard approach to recording or reporting on 
complaints. Data cannot be aggregated beyond each organisation to identify 
emerging trends in complaints, or analyse the user's perspective”. But to get the 
greatest insights from complaints data, we must also look beyond formal 
complaints to less formal routes and outlets. 


Box 1. The value of aggregation: Identifying policy impacts in advice trends 


At Citizens Advice, we understand the value of data aggregation. Our dedicated 
volunteer advisers helped 2.5 million clients last year, providing Support, advice 
and advocacy at more than 2,500 local Citizens Advice offices and outreach 
locations on more than 6.2 million advice issues. 


The ability to record and aggregate this detailed advice issue data across the 
service provides us with a unique breadth and depth of insight into the issues 
people face. It also means we are able to identify emerging issues across a range 
of social policy areas and point to potential policy solutions. 


For example, nationally aggregated advice query data from local Citizens Advice 


42 Citizens Advice client and interviewee. 
43 NAO (2015), Public service markets: Putting things right when they go wrong. 
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offices across the country recently alerted us to an upsurge in Council Tax debt 
queries. Since the Government's replacement of Council Tax Benefit with a local 
model of Council Tax Support in 2013, Council Tax arrears have overtaken credit 
card debt as the number one debt problem our clients seek help with. This has 
given us significant insight into the challenges of the current Council Tax Support 
model, as well as some examples of better practice, and has fed into our 
response to the Government review into Council Tax Support, launched in 
December 2015.4 


Figure 10: Council tax debt and credit card debt advice queries 
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Case studies from social media: Exploring how complaints data can be 
identified and aggregated 


In this section, we look at social media as a potential new source of rapid 
intelligence on lower level complaints and we use this as a case study to explore the 
wider benefits of aggregating complaints data. 


The best of the private sector is acutely aware of the value of social media in 
improving customer experience. The Institute of Customer Service suggests that 
“one in four social media users in the UK used platforms such as Facebook, Twitter, 
Instagram or Google+ to make a complaint over the past 3 months”.® The Office for 
Rail Regulation recently highlighted good practice by train companies like Greater 
Anglia, which uses Twitter complaints intelligence to “construct a daily log of 
categories of passenger issues” SO managers can see issues faced by passengers 
‘on the ground’, and London Midland, which responds to every tweet directed at it 


44 Citizens Advice (2016), Review into local council tax support: Citizens Advice response. 
45 Institute of Customer Service (2015), Customer complaints see 8-fold rise on social media. 
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and “welcomes the benefits of a quick, manageable message size” and the “quicker 
identification of issues” it enables.*° 


Some public service providers are taking steps in this direction. The DWP has stated 
that making “better use” of big data is core to its “vision to create more efficient 
services”,*” and most recently the Chair of the Care Quality Commission (CQC) 
highlighted social media complaints data as a potential source of “early intelligence” 
on emerging service issues.* If, like the best of the private sector, public services 
like HMRC or the NHS, could collect and interpret feedback received through social 
media, it might be possible to better identify problems with their services as they 
emerge, and to identify suitable solutions. 


Social media as an informal channel for complaints 

Our Complaints Survey revealed that one in ten people (11 per cent) turn to social 
media after poor public service experiences. The figure nearly doubles, to one in 
five (19 per cent) amongst people in the 18-24 age bracket. 


Figure 11: Proportion of people who complained about poor public service on 
social media, by age group® 
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As tomorrow's middle aged people, younger people provide us with insight into 
how people’s preferences overall are likely to shift over time. While social media 
currently appeals more to those in the 18-24 bracket we can anticipate that usage 
of this complaints channel will grow in popularity.°° 


Tweeting complaints at public services 


46 ORR (2014), Passenger engagement: How train operators listen to their passengers. 
47 DWP (2016), The appliance of data science. 
“8 Laura Donnelly (2016), NHS will monitor Facebook for comment about hospitals, says new CQC head. 


“° Responses based upon the 11 per cent of those Complaints Survey respondents who said they 
“spoke about my experience on social media (e.g. Facebook or Twitter)” after their last poor public 
service experience. 

°° See Ofcom (2015), Adults’ Media Use and Attitudes. 
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We chose to look at three services to explore the possible benefits of social media 
analysis. Using natural language processing to design a series of algorithms, we 
examined all tweets over the course of a year directed at HM Revenue and Customs 
(HMRC), the Driver and Vehicle Licensing Agency (DVLA) and 345 local authorities 
across England. Our research uncovered a total of 114,273 complaints directed at 
these services.°' Our aim was to understand how many of these tweets could be 
classified as complaints, and what sorts of public service issues people complain 
about on social media. 


Case study finding 1: Lots of complaints are made through social media 

Our research identified 114,273 complaint tweets aimed at just three public 
services over the course of a year. We found that both HMRC and DVLA’s twitter 
accounts attract more complaints than tweets of any other kind. Nearly six out of 
ten tweets (57 per cent) directed at HMRC represented a complaint and over halt 
(54 per cent) of tweets mentioning DVLA were complaints. 


Figure 12: Proportion of complaints amongst all ‘at mentions’ of public 
services”? 
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Case study finding 2: Most complaints through Twitter are about low level 
frustrations 

We found that people typically use Twitter to register low level service complaints 
relating to communication failure (whether over the phone, in writing, online, or 
face-to-face), administrative errors, delays, and other everyday frustrations. People 
were less likely to use Twitter to complain about ‘high level’ policy issues or very 
serious ‘life and death’ service issues. 


Figure 13: Examples of tweeted public service complaints 


Hey @lambeth_council, we're trying to recycle @aylesburyvale Your new website seems to 
but you haven't posted us recycling bags and purposely block people from finding a no. to 


none in Town Hall/Customer Centre either contact you by phone. Disappointing as a local 
9:40 AM - 27 Aug 2015 service 


>1 See Methodology for details. 
>¢ Based upon NLP analysis of 441,172 tweets ‘at mentioning’ these services between August 2014 and 
August 2015. 
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@DVLAgovuk Does nobody there have an @DVLAgovuk Your website is down... 
improvement mentality? 

The last two times | reported problems with the 

web site | just get stock reps 


Nightmare getting through to @HMRCgovuk! @HMRCgovuk 52 minutes on hold just to be 
On hold for 45 mins. 'One of our advisers will cut off & 

be with you as soon as possible’. 

Hmm...#frustrated 


1 R 


The vast majority of complaint tweets directed to HMRC and DVLA are about low 
level service problems. 78 per cent of all complaints directed at HMRC, and 82 per 
cent of those directed at DVLA, were about these service issues. 


It was harder to generate such a distinction when examining local authorities.” 
However, 22 per cent of complaint tweets were about waste and cleanliness issues, 
and three per cent were about parking. We would generally classify these as low 
level service issues too. 


Our findings here are perhaps not surprising. It can be difficult to know where to 
turn when you experience poor low level or ‘customer service from a public service, 
as opposed to a more serious or high level failure. 


Case study finding 3: Social media data aggregation can help identify specific 
and ‘real time’ emerging service issues 

12,588 complaints directed at HMRC were about their telephone helpline services; 
that amounts to more than one in three (37 per cent) of all tweets mentioning 
them. Of these helpline complaints, 91 per cent - 11,465 tweets - were about time 
waiting on hold or for a call to be picked up. This shows how aggregating 
complaints data could help inform service improvements; had HMRC monitored 
this issue as it emerged in real-time on social media, it may have been able to 
introduce more call handlers at an earlier stage.” 


Similarly, 1,594 tweets directed at the DVLA were about service user frustrations 
with the agency’s website or e-mail service. DVLA could systematically monitor this 
data, enabling the quick reporting of downtime and other issues to its web team to 
achieve a speedy resolution. 


Studying tweets mentioning local authorities, which are responsible for delivering a 
much wider range of services than HMRC and the DVLA, we found that around half 
of all the complaints about waste and cleanliness were specifically about domestic 
refuse collections. Local authorities could be using this data to find out when and 


3 See Methodology for discussion of the limits to categorising data using machine learning techniques. 
> HMRC announced in June 2015 that thousands of staff were being recruited to its customer service 
teams to improve telephone helpline services. 
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where collections have been missed, or incorrect bins supplied, so they can quickly 
remedy these issues before people become frustrated. 


While our analysis looked at static, retrospective data, it is possible to apply these 
NLP algorithms on an ongoing basis, collecting and analysing live data. This would 
allow for frequent checks, in real time, on emerging complaints themes and the 
service issues causing complaints at a given time.” 


For example, we found that far more complaints about telephone helplines are 
directed at HMRC in the first two weeks of each month, as Figure 14 shows. While 
there are many possible explanations for the pattern - from people’s behaviour 
around tax issues to the dates of tax credits payments - the solution is clear; HMRC 
needs increased staffing and capacity on its helplines in the first half of each 
month. 


Figure 14: Telephone helpline complaint tweets @HMRCgovuk, by day of the 
month, aggregated across 12 months”° 


Telephone helpline complaint tweets per day 
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Apart from collecting data on a single service like HMRC or the DVLA, social media 
analysis allows for the aggregation of complaints across services. We were able to 
use our data on complaints about domestic waste collection issues to produce the 
map in Figure 15, which shows where these problems are most acute. 
Unsurprisingly, complaints are heavily clustered around population centres such as 
London, Birmingham, Manchester and Newcastle, but smaller clusters of 
complaints about these waste issues can be identified elsewhere in England. 


> The ability to identify real time emerging issues and trends from our clients’ advice queries is central 
to our 

> Based upon NLP analysis of the 64 per cent of all complaints tweets mentioning HMRC that were 
about telephone helplines. 
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Figure 15: Map of tweets complaining to local authorities in England about 
domestic refuse collection?” 
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Box 2. Good practice case study: Local authorities 


Some public service providers are leading the way in terms of good practice, 
using social media, crowd-sourced and aggregated data to improve their services. 


Of 345 English local authority Twitter accounts we looked at, nearly one in two (44 
per cent) takes a proactive approach, explaining in their profile how and when the 
account is monitored, and in some cases positively encouraging feedback. Local 
authorities such as Basildon and Brentwood state they will respond to all queries 
from the public within 48 hours, with named staff signing-off messages. However, 
the approach is uneven, with other local authorities effectively using their Twitter 
account for ‘broadcast’ only. 


Some local authorities use ‘crowd-sourced’ data from the public to log and 
respond to local problems like potholes or fly-tipping. Websites and mobile apps 
like FilllhatHole (run by cyclists’ advocacy group CTC) allow users to plot potholes 
on a map, automatically informing the relevant council, while Glasgow City 
Council's ‘MyGlasgow’ app allows users to log a range of unique issues - from 
street lighting to parking - on a map, avoiding duplicate recording.”® 


Local authorities working closely with local Citizens Advice offices provide another 
example of good practice. For instance, following a number of complaints about 
housing benefit administration, Southwark Council worked with its local Citizens 


>’ Based upon NLP analysis of the 50 per cent of all waste complaint tweets mentioning local 
authorities that were about domestic refuse service issues. 


8 LGIU (2014), Technology and transformation in town halls. 
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Advice to explore how to deal with debt and vulnerable clients. This close working 


has enabled the council to learn from the feedback we provide on behalf of our 
clients and use that insight to improve services. 


The aggregation and analysis of complaints data from social media in our case 
studies highlights the potential for public service complaints aggregation more 
broadly. By collecting, interpreting and acting on real time feedback from service 
users, public services can become more responsive and efficient, and the 
experience of users can be improved. 
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Chapter 5. Towards more open 
and responsive public services: 
Recommendations and 
opportunities 


The current systems in place for capturing public service complaints are in need of 
urgent overhaul. Important changes are already underway, and we welcome the 
Government's proposals to simplify the ombudsman system for handling top level 
formal complaints. But bringing together the public service ombudsmen alone will 
not solve many of the issues we see people struggle with when they navigate 
complaints processes. Through our research, we identify a number of important 
opportunities: 


@ There should be no ‘wrong door’ when people want to make a 
complaint. All types of complaints offer valuable insight, but not every 
complaint can go through formal processes or to an ombudsman. The 
Government should ensure we capture complaints from everyone: 

O Certain groups and individuals don't feel that they have the necessary 
resources - be that time, Knowledge or social capital - to go through 
the formal complaints processes. By improving and broadening the 
range of complaints channels people can use, we can enhance our 
understanding of service failings and accelerate improvements. This 
Should include removing unhelpful layers, such as the ‘MP filter’ on 
non-health complaints to the PHSO. 

O Some types of complaint are not serious enough - in terms of 
individual service user detriment - to warrant the time and effort 
needed for a formal complaint, but must nevertheless be captured 
and acted on by public services where their volume indicates an 
emerging trend or problem. This requires a widening of what services 
consider ‘legitimate’ complaint routes. 


@ Public bodies should develop better insight and intelligence capabilities 
on emerging service issues through big data aggregation. Aggregating 
data across services allows for the development of a more complete national 
picture of emerging issues. New communications technologies and digital 
social networks present a real opportunity to draw upon as yet untapped 
pools of complaint data that should be fed back into service design. Our own 
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analysis highlights how social media can offer insight into potential service 
improvements, allowing the real time identification of service problems - 
from helpline overload and website downtime to missed refuse collections 
and administrative errors. Monitoring social media complaints means 
working with how people actually do complain, rather than how services 
might want them to. 


Public service providers need to modernise their complaints culture 
and empower frontline staff. Too often, as our polling and interviews 
show, service users are left feeling that public service providers either won't 
respond or will ‘close ranks’ when faced with a complaint. Frontline public 
service staff must be enabled to handle more service issues directly; only one 
in five (19 per cent) of people told us they did not pursue a formal complaint 
because they were able to resolve their complaint without needing to do this. 
This will enable ongoing learning and improvements within public services. 


People should be able to complain to a single point of contact. Our 
research highlighted the difficulties people face understanding whether and 
when to escalate their complaint. Public service ombudsmen are forced to 
turn away a significant proportion of the enquiries they receive or refer them 
back to ‘local resolution’. A triage system would provide a ‘one-stop shop’ to 
capture and categorise people’s complaints, directing to the right place, 
whether local resolution or an ombudsman. This system would improve the 
customer experience, reducing costly duplication within the system, while 
also providing further useful data on public service performance. 


People should have access to an independent, impartial organisation 
for help with their complaint. One in four people (26 per cent) told us they 
would be more likely to complain about poor public service with ‘support 
from an advice or advocacy organisation’, a figure that rises to nearly one in 
three (32 per cent) among people aged 65+. Some potential complainants 
will be reliant upon the public service in question for their health and 
wellbeing, and will therefore be concerned about complaining without 
independent support. One in five non-complainants (19 per cent) cite fear of 
reprisal as a reason for not complaining. This anxiety is especially 
pronounced amongst older people and those on lower incomes. 


The power to raise ‘super complaints’ on behalf of public service users 
should be introduced. In the private sector, super complaints - like the one 
Citizens Advice raised on the mis-selling of payment protection insurance 
(PPI) - have been instrumental in achieving sector-wide reforms for the 
benefit of all consumers by drawing attention to systemic problems. Public 
service users must also be afforded this high-level representation, advocacy 
and redress to improve service quality. 
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Free, confidential advice. 
Whoever you are. 


We help people overcome their problems and campaign on big issues when their 
voices need to be heard. We value diversity, champion equality, and challenge 
discrimination and harassment. We're here for everyone. 


Ben Whitham, Temi Ogunye and Lizzie Greenhalgh. 


citizensadvice.co.uk 
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